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ABSTRACT 

This article deals with a subject that is extremely important to the success of future graduates of 
any college or University - specifically Critical Thinking and Decision Making. Our article 
explains the research results and observations of critical thinking research conducted at two 
different colleges in the fall semester of 2011. The research prompts were used at two different 
colleges (Wagner College and Monmouth University) with different student profiles during the 
same semester. The purpose of the research was to test how critical thinking skills were affected 
by two different experiential programs and how “Ex-Ed" is a critical part of a student’s total 
education. 

Experimental Education (Ex-Ed) is a very hot topic among academics these days and our research 
at Wagner College involved testing the improvement in critical thinking skills in a “marketing 
incubator” system during a Consumer Behavior class with 34 registered students in the fall 
semester in 2011. In this case, some of the students participated in the marketing incubator (18), 
while some did not (16), essentially providing a control groups for comparison. At Monmouth 
University, the research was testing the improvement in critical thinking skills in two 
entrepreneurial studies classes with a total of 67 students. Qualitative observations are included 
as the research is conducted, and suggestions for future related research are proposed. Our 
feeling is that to form a definitive conclusion, the subsequent research needs to be done in the 
area of both decision making and the creation of a value proposition. 

Keywords: Critical Thinking; Experiential Learning 


INTRODUCTION/LITERATURE REVIEW 


ritical thinking (CT) has been defined as “the use of those cognitive skills or strategies that increase the 
probability of a desirable outcome” (Halpern, 1998, p. 450). According to Roland Case, “every 
curriculum document mentions critical thinking, and there is universal agreement about the need to 
make thoughtful judgments in virtually every aspect of our lives- from who and what to believe to how and when to 
act” (2005, p. 45). 



Declining American test scores are directly related to the lack of critical thinking skills being learned in our 
Colleges and Universities. “The U.S. Department of Education has linked the decades-long decline in education 
outcomes to students’ poor Critical Thinking Skills” (Reid, 2010, p. 7). Winn emphasizes the failures in teaching 
critical thinking (2004) and Case (2005, p. 45) stated that “he is disheartened by the failure to teach Critical 
Thinking”. Willingham (2007, p. 8) goes a step further by asking, “Can critical thinking be taught?” Rhetorically, he 
answers, “Not really.” 


Several different sources of constructivist learning such as Problem Based Learning, Case Study Analysis, 
Internships and service experiential learning have been tried in order to improve the critical thinking of students. 
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For example, Carini, Kuh and Klein (2006) measured student engagement in 14 four year colleges and 
universities measuring many project based learning techniques. Zigler (1994) presented a single universal, 
unverified case study in which new administrators were taught skills based on experiential learning. McCormick 
(1993) claimed increases in Critical Thinking based on the experiences of interns. In a report from the Office of 
Educational Research and improvement, Stein (2000) claims that critical reflection combines experiential learning 
with technical learning to create new constructs of knowledge, new behaviors, or new insights, leading to Critical 
Thinking. Jakubowski (2003) claims that a field trip to Cuba encouraged Critical thinking. Abrami et al. (2008) did a 
study measuring a whole host of CT studies in order to identify a pattern of success. 

Despite the number of studies, there are still mixed results as it relates to linkage to students’ Critical 
Thinking, measurement methods. 

This study evolved because Professor DeSimone of Wagner College and Professor Buzza at Monmouth 
University share the belief that experiential learning is the best way to improve the efficacy of undergraduate 
business education by improving critical thinking skills for this generation of students. However, they both 
independently developed significantly different experiential systems to apply this experiential learning component in 
their specific classes and institutions. This research study attempts to use a similar CLA Critical Thinking essay 
prompt to measure the linkage between the two. 

Professor DeSimone is applying what he refers to as a “marketing incubator” program where students 
registered for core marketing classes are given an option to either participate in the “marketing incubator” or to not 
participate in the marketing incubator, and take the class with a traditional final. In the marketing incubator, the 
class is given an opportunity to interact with a “real world” company and aid them in some portion of their 
marketing function. The assignment includes the negotiation of a “scope of work” or project definition as to what 
the student team will have to complete by the end of the semester. The student team then acts as a “mock advertising 
agency” with Professor DeSimone as the account executive. The business person in the real world company 
identified real world marketing challenges, and acts as the client to the advertising agency. In this education model, 
the students have exactly the same class time, syllabus, and assignments except that of their final exam. The 
participating group (experimental group) work throughout the semester with the assigned “real world” client. The 
control group takes a traditional take home written final exam. The participating students are graded based on the 
quality and creativity of their marketing deliverables as compared to their agreed scope of work, and will make an 
informal presentation comparing the original “scope of work” to what they achieved and delivered for the client. 
The control group will also make an informal presentation at the end of the class, selecting one of the topics from 
their take-home final to present to the class in detail. 

Some examples of specific deliverables from the marketing incubator teams in fall 2011 are event planning, 
research, the construction of a social media plan, and creative planning for print advertising. 

Professor Buzza is the Director of the Center for Entrepreneurial Studies at Monmouth University and is 
applying his experiential component to two classes entitled Entrepreneurship and Small Business Management in 
the fall 2011 semester. In this form of experiential learning, the students all participate in the start-up of an 
entrepreneurial venture, encompassing all phases of the operation including legal, finance, marketing, and 
management. The students create the company with the understanding that they will be able to “buy” the company 
back at the end of the semester for $1.00. In this experiential format, the students are broken into teams, and 
assigned specific tasks and timelines in order to accomplish the launch within the semester. In this case. Prof. Buzza 
plays the role of CEO, and guides, teaches and aides all of the students in their assigned tasks throughout the 
semester. At the end of the semester, the students do a formal presentation describing the new company and the 
process they went through to develop the company. 

In the fall 2011 semester, the students and Prof. Buzza created and presented a GPS Tracking company. 

Starting in the fall of 2011, Professor DeSimone and Professor Buzza designed a joint research study to test 
the impact of their program on the critical thinking skills of their students. The purpose of the research was to 
measure if students’ critical thinking skills were being affected by the two different experiential components 
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embedded separately in their programs. The co-authors also want to begin the process of qualitatively assessing 
which of the experiential components mix and match to provide the most efficacious student experience. 

This article analyses the research results and observations of critical thinking research conducted at Wagner 
College and Monmouth University in the fall semester of 2011. 

Study Design 

The same Critical thinking research prompts were used at two different colleges with two different 
experiential programs with different student profiles during the fall 2011 semester. 

The research tested the improvement in critical thinking skills in the “marketing incubator” experiential 
method described above during a Consumer Behavior class with 34 registered students. In this experimental design, 
some of the students elected to participate in the marketing incubator (18), while some did not (16); in effect, 
providing an experimental group and a control group for direct comparison between the participants and the non¬ 
participants in the experiential component. At Monmouth University, the research was testing the improvement in 
critical thinking skills in two entrepreneurial studies classes with a total of 67 registered students. In both cases, the 
students were given a series of critical thinking prompts near the beginning of the semester and a series of critical 
thinking prompts near the end of the semester. The critical thinking prompts were then scored by the same two 
Wagner College Graduate Assistants (GA’s), the scores of the two GA’s were averaged to reduce bias, and the 
average mean prompt scores were tabulated. Then the beginning-of-the-semester prompt scores were compared to 
the end-of-semester prompts scores for each school, and a one and two-tailed t test was run assuming unequal 
variances to test for statistical significance. 

HYPOTHESIS 

The author designed the original research to measure the growth in critical thinking scores in courses at 
Wagner College in Staten Island, New York, designed to embed an experiential component (the marketing 
incubator) into the course structure. 

1. Those students participating in the study (experimental group) will show statistically significant 
improvement in average prompt scores as compared with those students that did not participate (control 
group) within a confidence level of 95%. 

The author applied the same type of research prompts to measure the growth in critical thinking scores in 
Entrepreneurial studies courses at Monmouth University using an entrepreneurial start-up as its experiential 
component. 

2. The students taking this course will show statistically significant improvement in average prompt scores 
from the beginning of the semester to the end of the semester within a confidence level of 95%. 

WAGNER CONSUMER BEHAVIOR (MK301 COURSE, FALL 2011) 

Quantitative Research Analysis - Student Profile 

Thirty four students registered for MK 301, Consumer Behavior for the fall 2011. Below are the 
demographics of this class: 

• It is a marketing core course where majority of the students have a concentration or minor in marketing. 

• Thirty-four students were registered. 

• Twenty-nine students were seniors and five were juniors. 

• Twenty-two were business administration majors, ten were Arts Administration majors, one was a Theatre 
Major and one was a Psychology major. 

• Twenty-four students were female and ten were male. 

• Approximately 75% of Wagner students live on campus; 25% are commuters. 
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Procedure 

On the first day of class the instructor of the class covered the course syllabus. As stated on the course 
syllabus, the students were given an option to participate in the marketing incubator project as their final 
requirement, or to do a six-page final written exam. The written exam assignment on the syllabus was: 

“To choose three topics from the text of interest to them, and expand upon that topic by doing additional 
research, applying that theory to their experience, or otherwise dispute or expand upon the theory”. They would also 
have to select one of the topics they chose and present that topic to the class. 

For both groups, the final was worth 20% of their final grade. Those that chose to participate in the 
incubator were also told they could earn extra credit of 10% of their final grade if they were to exceed the “scope of 
work” agreed with the client. This was also meant to compensate the students for additional time outside of class 
time for group meetings with the client, and separate group meetings to coordinate their workload and collaborate on 
the “scope of work”. To avoid any bias, students that chose not to participate in the marketing incubator were also 
able to obtain extra credit through various traditional assignments such as case study analysis or an extra paper. Of 
the total of 34 students registered in the class, 18 students chose to participate in the marketing incubator project, 
and 16 chose not to participate in the marketing incubator project. Below is table 1 which lists the 5 type of 
incubator clients and the number of students selecting each one in the fall 2011 consumer behavior class. 


Table 1: Students by Incubator Client Type 


Type of Incubator Client 

Number of students in the team 

Sports Marketing 

4 

A Catering Company 

4 

Government/College Partnership 

4 

Environmental Marketing 

3 

Service Marketing 

3 


Research Methodology 

All students were given the survey shown in Appendix A on September 20. The Instructor of the course 
authored the survey including two critical thinking prompts. In an effort to make the prompts general enough to 
foster critical thinking, the Instructor borrowed quotations from the article Advertising: The Poetry of Becoming by 
Theodore Levitt (1993). This article was not part of the course work. This prompt was reviewed by Dr. Lo Re, the 
Chair of the Department of Business Administration at Wagner College and subsequently again by the Center for 
Teaching, Learning and Research at Wagner College chaired by Katia Gonzalez. 

The students were given exactly 20 minutes to complete the survey. They were also given the typical blue 
booklets to write their answers. They were told not to put their names on the survey and to do their best according to 
the instructions on the survey itself. 

Of the 34 students registered in the class, 32 were present on September 20 to take the survey and answer 
the essay prompts. The survey was given at the beginning of the class after attendance was taken. After the students 
completed the 20 minute survey and essay prompts, the booklets including the prompt responses were collected and 
given to the two business department graduate assistants (GA’s). The graduate assistants were given a short 
background about the study, and a copy of the survey and essay prompts and told to assign the anonymous booklets 
a number from one to 32. Therefore, each of the two graduate assistants was independently scoring booklet number 
1 and then number two and so on. Consequently the score for number one for each GA’s was for the same booklet. 
This later proved to be useful when we needed to reconcile a discrepancy in the scoring of N/A (not applicable) if it 
occurred. The GA’s are Graduate Business Administration students of Wagner College and for the fall 2011 
semester are both accounting majors. The GA’s were then given the Teagle scoring grid titled “SOLO Taxonomy 
for Assessing Level of Critical Thinking”. (Appendix B) More details can be found on multiple websites. One such 
website is: http://www4.ncsu.edu/~damcconn/ct2 background.html (McConnell, n.d.). This method of scoring was 
chosen because this is the standard Critical Thinking scoring technique used at Wagner College to score overall 
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College Collegiate Learning Assessment (CLA) exam results for the college as a whole. The Instructor of the 
course. Prof. DeSimone, is modestly acquainted with this method because of his participation in the Freshman, 
Intermediate and Senior Learning communities at Wagner College. He moderated a number of CLA exams, and 
received research analysis and results as part of the Intermediate and Senior Learning Committees. He also attended 
a presentation on the topic by an expert in the field at Wagner College, Dr. Donald Sterns, on February 10, 2011 
about the methodology. 

The other reasons why this particular measurement tool was chosen are: 

1. These (CT) rubrics are widely used amongst civic engagement practitioners and particularly by Wagner 
College 

2. Since the research prompts can be designed for a particular class, it is hopefully more motivational for the 
student to apply critical thinking 

3. The essay prompt can be made more specific to the subject matter 

4. There is no cost or budgetary concerns 

5. It is easier to scale to other classes and other institutions 

6. The CLA is an accredited assessment organization 

7. It is the measurement tool of choice for Academically Adrift (Arum & Roksa, 2011), that was an inspiration 
for much of the author’s research into designing this research. 

The booklets were then scored, and the scores were compiled on an excel spreadsheet by the GA’s and then 
the scores were averaged in order to reduce scoring bias. A summary of the scores are seen in Table 2 below: 


Table 2: Summary of Survey Scores 


Consumer Behavior t-tests - Two Samples Assuming Unequal Variances 

All Students (n=32) 

Beginning of the Semester 
September 20, 2011 

End of the Semester 
December 8, 2011 

Improvement 

One- 

tail 

Two- 

tail 


# Observations 

Mean 

Prompt 

Score 

# Observations 

Mean 

Prompt 

Score 


t-test 

t-test 

Prompt 1/P2 

32 

3.83 

32 

3.91 

2.04% 

0.3934 

0.7868 

Prompt 2/PI 

28 

3.20 

31 

3.79 

18.58% 

0.0490 

0.0800 

Average combined 


3.52 


3.86 

9.80% 

0.1047 

0.2094 


Consumer Behavior t-tests - Two Samples Assuming Unequal Variances 

Participating 

(n=18) 








Prompt 1/P2 

18 

3.89 

17 

4.26 

9.66% 

0.1918 

0.3835 

Prompt 2/PI 

15 

3.47 

16 

4.09 

18.09% 

0.0561 

0.1121 

Average combined 


3.68 


4.07 

10.689% 

0.1354 

0.2707 


Non-Participating 

(n=16) 








Prompt 1/P2 

14 

3.75 

15 

3.50 

-6.67% 



Prompt 2/PI 

13 

2.88 

15 

3.47 

20.18% 



Average combined 


3.34 


3.48 

4.31% 

0.3772 

0.7548 


Analysis of Findings 

For the beginning-of-the-semester research. Prompt 1 scored an average mean of 3.83 out of a maximum of 
7.0 with a variance of 1.01 with 32 observations. Prompt 2 scored an average of 3.20 out of a maximum of 7.0 with 
a variance of 2.10 with 28 observations. The difference in observations is attributable to those that did not complete 
an answer to a prompt. The GA’s were instructed to disregard any answer that was not complete and assign an N/A. 
In the case where an answer was judged to be incomplete by both GA’s independently, it was scored as N/A and not 
averaged into the mean score. If one GA did provide a score, and another did not, then a third GA would review 
both booklets and would make the decision to score the answer or not. 
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This same methodology was repeated again with the same GA’s as scorers with the same instructions by 
giving the survey/prompts (Appendix C) to the students of the consumer behavior class on December 8, 2011. 

The scores were again put on an excel spreadsheet by the GA’s and then the scores were averaged in order 
to reduce scoring bias. The end-of-semester average of Prompt 1 scored an average mean of 3.79 out of a maximum 
of 7.0 with a variance of 1.10 with 31 observations. The end-of-semester Prompt 2 scored an average of 3.91 out of 
7.0 with a variance of 1.64 with 32 observations. Again, the difference in the number of observations was due to the 
application of N/A to certain responses. 

As part of the end-of-semester research, the prompts were reversed. In other words, the prompt 1 survey in 
the beginning of the semester is actually prompt 2 at the end of the semester and the Prompt 2 survey in the 
beginning of the semester is the same as Prompt 1 at the end of the semester. 

If we were to change the prompts, we would not be able to ascertain if the average scores were different 
because the prompts were easier or harder to critically review. The rationale for this decision is that by inverting the 
prompts, we would eliminate this possibility. 

The beginning-of-the-semester prompt 1 had an average mean score of 3.83. while the end of the semester 
score on Prompt 2 was 3.91, or a 2.04% improvement in the end-of-semester critical thinking scores as compared to 
the beginning of the semester. The two-tailed t-test results showed 0.79 significance, or an insignificant result for 
this prompt comparison. 

The beginning-of-the-semester prompt 2 had an average mean score of 3.20. while the end of the semester 
prompt 1 had an average mean score 3.79, or a 18.58% improvement of the end of the semester mean scores over 
the beginning-of-the-semester scores. The two-tailed t-test showed a .08 significance, or a significant result within 
8% variance for this critical thinking prompt score. 

The average of the mean of both prompt 1 and 2 in the beginning of the semester was 3.52, while the 
average of the means of the end of the semester scores was 3.86, or a 9.8% combined improvement in the average 
scores at the end of the semester mean prompt scores as compared with the beginning-of-the-semester mean prompt 
scores. The variance of the scores compared to the mean in the beginning of the semester was 1.18 and the variance 
of the scores compared to the mean at the end of the semester was 1.17. While the overall improvement in the 
student average mean prompt scores was 9.8%, the two-tailed t-test results indicated a 0.21 significance, or, a 
statistically insignificant result. 

As it relates to the important question of the difference in the scores of those that did participate 
(experimental group) in the marketing incubator program as compared to those that did not (control group), we 
separated the responses accordingly. 

The first observation as it relates to those students that participated in the incubator project as opposed to 
those that did not, is that the beginning scores are higher for those that chose to participate in the marketing 
incubator project by a considerable margin. For example, the average beginning combined score for those that 
participated was 3.68 as compared with an average beginning prompt score of 3.34 for those that did not, or a mean 
score that was 10% higher to begin with. Considering they are coming from the same class in the same school with a 
very similar demographic and registering with the same professor, the authors believe that is a considerable 
difference to start with and deserves future study. This may imply something about the critical thinking skills of 
those students who are more interested or willing to participate in an experiential “real world “exercise in the first 
place. More will be discussed about this result in the observation and analysis section of this paper. 

For those that did participate in the marketing incubator the beginning-of-the-semester prompt 1 had an 
average mean score of 3.89 while the end of the semester score on Prompt 2 was 4.26, or a 9.66% improvement in 
the end-of-semester critical thinking scores as compared to the beginning of the semester. The two-tailed t-test 
results indicated a 0.38 significance, or an insignificant result for this prompt comparison. 
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The beginning-of-the-semester prompt 2 had an average mean score of 3.47. while the end of the semester 
prompt 1 had an average mean score 4.09, or a 18.09% improvement of the end of the semester scores over the 
beginning-of-the-semester scores. The two-tailed t-test showed a 0.11 significance, or a significant result with an 
89% confidence level. 

The average of the means of both prompt 1 and 2 in the beginning of the semester was 3.68, while the 
average of the means of the end of the semester scores was 4.07, or a 10.68% improvement in the average prompt 
scores at the end of the semester scores as compared with the beginning-of-the-semester prompt scores. The 
variance of the scores compared to the mean in the beginning of the semester was 1.52 as compared with 0.65 at the 
end of the semester. The two-tailed t-test results indicated aO .27 significance, or an insignificant result. 

For those that did not participate in the incubator, the beginning-of-the-semester prompt 1 had an average 
mean score of 3.75, while the end of the semester scores on prompt 2 were 3.5, or a 6.67% reduction in the end-of- 
semester critical thinking scores as compared to the beginning of the semester. The two-tailed t-test results indicated 
a 0.59 significance, or an insignificant result for this prompt comparison. 

The beginning-of-the-semester prompt 2 score had an average mean score of 2.88 while the end of the 
semester prompt 1 score had an average mean score 3.47, or a 20.18% improvement of the end of the semester 
scores over the beginning-of-the-semester scores. While the end-of-semester scores indicated a 20.81% 
improvement, the two-tailed t-test indicated a 0.30 significance, or a statistically insignificant result. 

The average of the means of both prompt 1 and 2 in the beginning of the semester was 3.34, while the 
average of the means of the end of the semester scores was 3.48, or a 4.31% improvement in the average scores at 
the end of the semester scores as compared with the beginning-of-the-semester scores. The variance of the scores 
compared to the mean in the beginning of the semester was 1.68 and the variance of the scores as compared to the 
mean in the end of the semester was 1.32. The two-tailed t-test results indicated a 0.75 significance, or an 
insignificant result. 

Specific Semester Observations/Explanations - Wagner College 

The compelling part of the Wagner College research derives from the comparison of those that participated 
in the marketing incubator, compared to those that did not. 

When comparing the scores of the end-of-semester prompts of those that did participate in the incubator 
with those that did not, the findings become statistically significant according to the two-tailed t-test within 11% and 
a one-tailed t-test within 6%. 

Those that did not participate in the marketing incubator scored an average mean of 3.47 on Prompt 1 at the 
end of the semester. Those that did participate in the marketing incubator at the end of the semester scored an 
average mean of 4.09 on prompt 1. This represents an 18.09% better score and a two-tailed significance within 10%. 

Those that did not participate in the marketing incubator scored an average mean of 3.50 on Prompt 2 at the 
end of the semester. Those that did participate in the marketing incubator at the end of the semester scored an 
average mean of 4.26. This represents a 21.17% better score despite an insignificant two-tailed significance result. 

Those that did not participate in the marketing incubator scored a total average mean of 3.48 on both 
prompts at the end of the semester. Those that did participate in the marketing incubator at the end of the semester 
scored a total average mean of 4.07 on both prompts. This represents a 16.95% better total average score for those 
that participated in the marketing incubator than those that did not with two-tailed t test significance within 11% and 
one-tailed t test significance within 6%. 

While the research consistently shows an impressive gain with all students in critical thinking prompt 
scores from the beginning of the semester to the end of the semester, the most statistically significant scores are as a 
result of the scores at the end of the semester between those that participated in the marketing incubator and those 
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that did not. Since all other components of the course are identical this seems to indicate that the participation with 
the marketing incubator program leads to an improvement in critical thinking with the students over the course of 
one semester. 

It should be noted again that the differences in observations in some of the research is attributable to the 
fact that some students either did not answer prompt 2 or did not answer sufficiently to merit a score. Whenever this 
happened, the GA’s were told to mark the score with a N/A. If both GA’s scored a particular booklet N/A, then it 
was left N/A and not included in the averages. If one GA scored an N/A and another did not, a third GA (Alayna) 
was asked to review that booklet and make a determination whether the booklet should be scored or left as an N/A. 
Discussion with the GA’s and the students seem to indicate that some of them ran out of time answering the 
prompts, and others became “less motivated” to write as time went on. It should also be noted that the students 
seemed to score better on the first prompt of the research. In this study, 23 students scored better on the first prompt 
than the second prompt, 4 scored the same and 5 students scored better on the second prompt than the first. This 
should be noted as one of the weaknesses of the research study. 

MONMOUTH UNIVERSITY ENTREPRENEUR PROGRAM 

Quantitative Research Analysis 

Sixty-seven (67) students in both our Entrepreneurship (39) and Small Business Management (28) Class 
took part in the study. On the first day of class, Prof. Buzza went over the syllabus as he normally does. Early in the 
semester, the students were told they would all have to participate in a research study Monmouth University was 
conducting in conjunction with Wagner College to ascertain if their critical thinking skills improved as a byproduct 
of taking the course. The format of the course is to garner ideas from the students to start a business, one using “real 
money” that comes from Monmouth University by way of a grant, and to make that business come to fruition by the 
end of the semester. The students are told they will then be given an opportunity to “buy back” the company for $1 
to continue to launch the company they started. 

Comparison of the Studies 

The two studies represent two different styles of embedding an experiential component into the curriculum 
as a way to improve undergraduate business student critical thinking skills. 

In the Monmouth University Entrepreneurial Studies program, all students participate in the experiential 
exercise of creating and starting up a new company. Therefore, there is no control group as there is in the marketing 
incubator experiential component at Wagner College. All of the students are participants in developing the product 
launch plan and the main part of their grade is the performance of each member of each team relative to the tasks 
assigned to them to achieve the entrepreneurial start-up. In the marketing incubator approach, the course is a 
traditional consumer behavior course with the exception of the marketing incubator, which is worth 20% of the 
grade. Professor Buzza acts as CEO of the new company and provides his experience and support to each individual 
group and to the entire class to meet deadlines and accomplish their tasks. All students participate in the selection of 
the product or service they will launch, and the final decision is mutually agreed by the students in the class and 
Prof. Buzza. In the case of the marketing incubator. Prof. DeSimone interviews a number of prospective potential 
marketing incubator clients, and the class chooses a company in an area of interest that appeals to them, or 
alternatively chose to not participate at all and take the traditional final exam. Prof. DeSimone most closely acts as 
the account executive to each of the marketing incubator teams and their clients. The student demographic was 
different in that the entrepreneurial class at Monmouth University. The class contained a mixture of business majors 
performing all of the business functions including finance, accounting marketing, business law, and so on. while the 
consumer behavior class was attended primarily by marketing majors and minors, and arts administration students. 
Monmouth University program had two classes with 67 students in total participating in the experiential component 
while the consumer behavior class was only one class and had 18 students participating in the marketing incubator 
experiential component. The Monmouth University students were given three Critical Thinking essay prompts while 
the Wagner College students were given a survey with three questions and 2 Critical Thinking essay prompts. Both 
were given exactly 20 minutes to complete their respective prompts. Both groups did a presentation at the end of the 
semester to reveal what they had accomplished. 
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Monmouth Student Profile 

Sixty-seven students registered for Entrepreneurial Studies in the fall 2011. Below are the demographics of 
this class: 

• It is a general business course where the majority of the students have a concentration in business. The 
majority of the class was management majors (77%), followed by marketing majors (21%) and accounting 
(3%). There was also one science major, one political science major, and one music major. 

• Thirty-four students were registered in Entrepreneurial studies and in small business 

• Fifty-one students were seniors and 16 were juniors. 

• Thirty-three students were female and 34 were male. 

• Approximately 72% of Monmouth students live on campus; 28% are commuters. 

Research Methodology 

All students were given the prompts (Appendix D) on Wednesday, October 12, 2011. Prof. DeSimone 
authored the survey including three critical thinking prompts. In an effort to make the prompts general enough to 
foster critical thinking, he borrowed general concepts from the introductory section of the textbook entitled Strategic 
Marketing (Cravens & Piercy, 2009). 

The students were given exactly 20 minutes to complete the survey. Most of them were given the typical 
blue booklets to write in; while some completed the survey using lined white note pad paper. They were told not to 
put their names on the survey and to do their best according to the instructions on the survey itself. 

Of the 67 students registered in the class, all 67 were present on Wednesday, October 12, 2011 to write 
their response to the three prompts. The survey was given at the beginning of the class after attendance was taken. 
After the students completed the 20 minute survey, the booklets including the prompt responses were collected by 
Prof. Buzza. The prompt responses were then given to Prof. DeSimone to be scored by the same two Graduate 
Assistants that were scoring his Wagner College Mk 301 research prompts. 

The scoring procedure was identical to that used for the Wagner Consumer Behavior class. The booklets 
were then scored by the Wagner GA’s, and the scores were compiled on an excel spreadsheet by Noreen and Justin 
and then the scores were averaged in order to reduce scoring bias. 

This same methodology was repeated again with the same GA’s as scorers with the same instructions by 
giving the essay prompts to the Monmouth students on December 7th as represented in Appendix E. The booklets 
were then scored and the scores were compiled on an Excel spreadsheet by the GA’s and then averaged in order to 
reduce scoring bias. A summary of the scores are seen in Table 3. 

The beginning-of-the-semester scores for Prompt 1 were scored an average mean of 2.90 with a variance of 
1.23 with 67 observations. Prompt 2 scored an average of 2.72 with a variance of 0.85 with 66 observations. Prompt 
3 scored an average of 2.44, with a variance of 1.23 with 64 observations. 

The scores were again put on a spreadsheet by Noreen and Justin and then the scores were averaged in 
order to reduce scoring bias. The end-of-semester average of Prompt 1 scored an average mean of 3.01 with a 
variance of 2.05 with 44 observations. The end-of-semester Prompt 2 scored an average of 3.16 with a variance of 
1.43 with 44 observations. The end-of-semester Prompt 3 scored an average of 2.39, with a variance of 1.29 with 33 
observations. 
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Table 3: Analysis of Findings 



Beginning of the Semester 

End of the Semester 


One-tail 

Two-tail 

Monmouth University 

#Observations 

Mean Prompt Score 

#Observations 

Mean Prompt Score 

Improvement 

t-test 

t-test 

All students n=67 
















Prompt 1/ Prompt 3 

67 

2.90 

33 

2.39 

-17.32% 

0.0203 

0.0407 

Prompt 2/Prompt 2 

66 

2.72 

44 

3.16 

16.16% 

0.0212 

0.0423 

Prompt 3/Prompt 1 

64 

2.44 

44 

3.01 

23.54% 

0.0111 

0.0222 









Average combined 


2.74 


2.94 

7.39% 

0.1555 

0.3109 


As part of the end-of-semester research, the prompts were reversed. In other words, the wording of prompt 
1 in the beginning of the semester is actually the same wording as prompt 3 in the end of the semester survey and the 
wording of prompt 3 in the beginning-of-the-semester survey is the same wording as prompt 1 in the end of the 
semester survey. Prompt 2 is identical in both the beginning of the semester and at the end of the semester. The 
rational for this decision is that by changing the prompts, the research would be subject to the criticism that the 
beginning-of-the-semester prompts were either easier or harder than the end of the semester prompts. As will be 
discussed later in more detail under observations, the growth in the scores of beginning prompt 1 to ending prompt 3 
indicated a reduction in the score of 17.32%, the comparisons of prompt 2 (which were identical) indicated a growth 
of 16.16% and the comparison of beginning prompt 3 to ending prompt one indicated a 23.54% growth in average 
scores. This substantiates the findings in virtually all the previous research that the students seem to obtain higher 
mean scores on the earlier prompts. 

The beginning-of-semester prompt 1 had an average mean score of 2.90, while the end of the semester 
score on Prompt 3 was 2.39, or a reduction of 17.32% in the end-of-semester critical thinking prompt scores as 
compared to the beginning of the semester. The two-tailed t-test results indicated a .0407 significance, or a 
significant result for this prompt comparison within a confidence level of 4%. This finding again confirms the 
pattern we observed that the students perform significantly better in the earlier prompts than the later prompts. One 
should also notice that the observations on prompt 3 declined from 44 in prompts 1 and 2 to 33 in prompt 3. This 
reflects the fact that the GA’s determined that 10 of the scores were insufficient to grade at all for prompt 3. Again 
observations from the GA’s were that many of the students simply ran out of time. 

The beginning-of-the-semester prompt 2 had an average mean score of 2.72. while the end of the semester 
prompt 2 score had an average mean score of 3.16, or a 16.16% improvement of the end of the semester mean 
prompt scores as compared to the beginning-of-the-semester mean prompt scores. The two-tailed t-test showed a 
.0423 positive significance, or a significant result within a 5% variance. It should be noted here that this 
improvement is reflected by the same prompt in the same order in the survey. 

The beginning-of-the-semester prompt 3 had an average mean score of 2.44. while the end of the semester 
prompt 1 score had an average mean score of 3.01, or a 23.54% improvement of the end of the semester scores over 
the beginning-of-the-semester scores, The two-tailed t-test showed a .0222 significance, or a highly significant result 
within a 2% confidence. 

The average of the mean of all three prompts in the beginning of the semester was 2.74, while the average 
of the means of the end of the semester scores was 2.94, or a 7.39% combined improvement in the average scores at 
the end of the semester as compared with the beginning-of-the-semester scores. The variance of the scores compared 
to the mean in the beginning of the semester was 0.77 and while the variance of the scores compared to the mean in 
the end of the semester was 1.23. While the overall improvement in the combined average student critical thinking 
prompt scores was an impressive 7.39% higher at the end of the semester as compared with the beginning of the 
semester, the two-tailed t-test results indicated a .3109 significance, or an insignificant statistical result. 
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Specific Observations/Explanations- Monmouth University 

Two of the critical thinking prompt comparisons indicated a statistically significant result (with one 
comparison showing a 24% improvement within a 2% significance variance), while the negative drop in one prompt 
score made the overall result impressive with a 7.39% total improvement, but still not statistically insignificant 
within a 5% confidence according to the two-tailed t test. 

Again, the mean essay prompt scores of critical thinking improved from the beginning of the semester to 
the end of the semester at Monmouth University. Two of the prompt scores showed a statistically significant 
improvement from the beginning of the semester to the end of the semester, which calls into question the use of 
three prompts at Monmouth University and the 20 minute time limit. One also needs to consider that while the 
author wanted the critical thinking prompts to be vague enough for all business students to be able to write freely 
about, the initial design was specifically toward a marketing student. Almost 80% of the total students in the 
Monmouth classes were not marketing students. 

Monmouth University had a shorter elapsed time between the Critical Thinking prompt essays (56 days) as 
opposed to the Wagner College elapsed time between the Critical Thinking prompt essays (78 days). Since the 
literature on the topic questions whether Critical Thinking can be taught at all (Willingham, 2007), it is quite 
challenging to assume it can be taught with measurement taking less than one full semester. 

FUTURE RESEARCH 

Professor Buzza and Professor DeSimone are continuing to conduct similar research in the spring 2012 
with their respective classes. In addition, in the spring of 2012, Professor Neilsen of Wagner College is also 
conducting the same research in the same course number as Prof. DeSimone using the same essay prompts. She is 
adapting the marketing incubator experiential component so that three companies are being studied concurrently as 
opposed to over the entire semester. It will prove interesting to draw comparisons about the efficacy of the three 
different programs involving approximately 100 students in the same semester. 

After the 2012 semester research is complete, a number of the research issues need to be revisited. 

One is the effectiveness of the prompts themselves. If they are redesigned, how would the researchers test 
for comparability? Whether the prompts are changed or not, should the students be given more than 20 minutes to 
complete the survey. Should there be 2 prompts or 3 prompts? While the improvement in the average test scores are 
impressive in both studies, can we expect statistical significance in critical thinking for students within an elapsed 
timeframe of 78 days in one case (Wagner College) and 56 days (Monmouth University) in another? Reid (2010) 
shows research that has proven to do so with distance learning during the course of a semester, but also cites many 
studies that have not proven effective in improving critical thinking skills in the course of one semester. 

CONCLUSION 

The research is encouraging in that it indicates an improvement in Critical Thinking scores from the 
beginning of the semester to the end of the semester for both the Wagner College and the Monmouth University 
studies, but without statistical significance with a 5% confidence level according to a two-tailed t test assuming 
unequal variances. In the case of Monmouth University, since all the students participate in the program, there has 
been no control group to assess the potential improvement in critical thinking skills of the typical Monmouth 
University student. However, in the case of the Wagner research, where they did have a control group that took the 
identical class with an identical syllabus and the same professor, the results of the mean scores for those that 
participated improved at the end of the semester with a mean score that improved by 16.94% within an 89% 
confidence level from those that did not participate. 

The research will need to be refined to assess further the optimal pedagogical choice of incubator client in 
the case of the Wagner experiment, or the appropriate start-up company in the case of the Monmouth University 
program. Abrami et al. (2008) puts forth the proposition that experiential components in measuring critical thinking 
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are not all equal. The type of experiential intervention and depth of the participation of the professor can be a major 
factor in the success of the program. 

While we will need to refine and expand the research to provide more definitive statistical results, the 
research conducted so far has encouraged Professor Buzza and Professor DeSimone that their respective forms of 
experiential learning are clearly improving the critical thinking skills of their students. 
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“Critical Thinking” Prompt for MK 301 Distributed to the Class on Tuesday, September 20, 2011 

1. Are you a participant in one of the experiential projects in lieu of the final written exam? (List with name 
assignments attached) 

Please circle: Yes No 

2. Critical Thinking is defined by Wikipedia as “purposeful reflective judgment concerning what to believe or 
what to do”. On a scale of 1 -7, based on the lectures, assignments, and text readings so far, do you believe 
the MK 301 Consumer Behavior course will improved your critical thinking skills, with one being the 
lowest and seven being the highest? 

1 2 3 4 5 6 7 

3. Creativity (according to Wikipedia) refers to the phenomenon whereby a person creates something new (a 
product, a solution, a work of art) which has some sort of value. On a scale of 1 -7, based on the lectures, 
assignments and text readings so far, do you believe that the MK 301 Consumer Behavior course will 
improved your creativity, with one being the lowest and seven being the highest? 

1 2 3 4 5 6 7 

In an article titled Advertising: “The Poetry? of Becoming” by Theodore Levitt (1993), the author puts forth 
the following comments about advertising: 

1) Of course, people put up with a lot - and understand that advertising is a price we pay for choice 
and free access. Things could be worse. They also know that advertising can help in many ways. It 
informs, entertains, excites and alleviates. Yes, it intrudes, but it also adds variety and changes the 
pace. 

2) Human behavior is almost entirely purposive. Products are tools people use to get results, to fill 
needs or solve problems that are not merely technical. A washing machine does not just clean 
clothes; just alleviate drudgery and heavy labor, just save time. It also creates opportunity to do 
other, more satisfying and perhaps more worthwhile things, to help one look, feel and be better. 
To raise one’s spirits, to help one become what one wishes to be. The same may be said of the 
personal computer, the tractor, the mutual fund and almost everything else. 

Please critically evaluate at least two different sides or viewpoints of the two issues above. Please write 
neatly in the booklet provided. This paper should contain an appropriate level of sophistication and critical 
thought. 
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Scoring Grid for the Teagle Grid 


Level Level Characteristics 

1 No understanding demonstrated. Response does not address the question or restates the question. 

2 Limited understanding of the topic. Response focuses on one conceptual item in a complex case 
and is not accurate or partially accurate. 

3 Limited understanding of the topic. Response focuses on one conceptual item in a complex case 
and is accurate. 

4 Understanding of several discrete components Response is a collection of multiple items that are 
not related within the context of the exercise. 

5 Understanding of several components that are integrated conceptually. Response may not 
prioritize information or be appropriate to the scale of the question. 

6 Understanding of several components that are integrated conceptually. Response prioritizes 
information and is appropriate to the scale of the question. 

7 Understanding demonstrated at a level extending beyond what has been dealt with in the question 
prompt. Response generalizes to situations beyond the scope of the question. 
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“Critical Thinking” Prompt for MK 301 (December 08, 2011) 

4. Were you a participant in one of the experiential projects in lieu of the final written exam? (List with name 
assignments attached) 

Please circle: Yes No 

5. Critical Thinking is defined by Wikipedia as “purposeful reflective judgment concerning what to believe or 
what to do”. On a scale of 1 -7, do you believe the MK 301 course has improved your critical thinking 
skills, with one being the lowest and seven being the highest? 

1 2 3 4 5 6 7 

6. Creativity (according to Wikipedia) refers to the phenomenon whereby a person creates something new (a 
product, a solution, a work of art) which has some sort of value. On a scale of 1 -7, do you believe that the 
MK 301 course has improved your creativity, with one being the lowest and seven being the highest? 

1 2 3 4 5 6 7 

In an article titled Advertising: “The Poetry’ of Becoming ” by Theodore Levitt the author puts forth the 
following comments about advertising: 

3) Human behavior is almost entirely purposive. Products are tools people use to get results, to fill 
needs or solve problems that are not merely technical. A washing machine does not just clean 
clothes; just alleviate drudgery and heavy labor, just save time. It also creates opportunity to do 
other, more satisfying and perhaps more worthwhile things, to help one look, feel and be better. 
To raise one’s spirits, to help one become what one wishes to be. 

4) Of course, people put up with a lot - and understand that advertising is a price we pay for choice 
and free access. Things could be worse. They also know that advertising can help in many ways. It 
informs, entertains, excites and alleviates. Yes, it intrudes, but it also adds variety and changes the 
pace. 

Please critically evaluate at least two different sides or viewpoints of the two issues above. Please write 
neatly. This paper should contain an appropriate level of sophistication and critical thought. 
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Monmouth University Entrepreneurship Program 
Professor John Buzza, Wednesday October 12 

Please respond to the following questions (prompts). Please answer neatly in the booklets/papers provided. Do not 

put your name on the booklets or on this paper. These prompts are being independently evaluated as part of a 

research study, the results of which may be used for a published article. 

Your responses should be thoughtful, clear, precise, relevant, and reveal your logic and depth. Where appropriate, 

include your core values about the topic. 

You will have approximately 20 minutes to respond to all three questions: 

1. Please discuss the impact of the macroeconomic environment on strategic marketing. The macroeconomic 
environment today may include the state of the world (or U.S.) economy, unrest in the Middle East, rising 
oil and food prices, the U.S. budget deficit, and the current political situation in America. 

2. How would you include ethics and social responsibility as part of your strategic marketing decisions? How 
important are your core values to the strategic decisions you make? 

3. How important do you think self-confidence, self- concept and/or self-esteem are to your ability to make 
effective strategic marketing decisions? 

Thank you for your participation! 
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Monmouth University Entrepreneurship Program 
Professor John Buzza, Wednesday December 7, 2011 

Please respond to the following questions (prompts). Please answer neatly in the booklets/papers provided. Do not 

put your name on the booklets or on this paper. These prompts are being independently evaluated as part of a 

research study, the results of which may be used for a published article. 

Your responses should be thoughtful, clear, precise, relevant, and reveal your logic and depth. Where appropriate, 

include your core values about the topic. 

You will have approximately 20 minutes to respond to all three questions: 

1. How important do you think the clarity of your “sense of self’ is to your ability to formulate and execute 
effective strategic marketing decisions? 

2. How would you include ethics and social responsibility as part of your strategic marketing decisions? How 
important are your core values to the strategic decisions you make? 

3. Please discus the impact of more general macro issues on the specifics of your strategic marketing plan 
and/or specific marketing promotional proposals. These general marketing issues might include upper 
management objectives, level of Bureaucracy of the organization, pace of the organization, level and type 
of employees, and overall budget constraints. 

Thank you for your participation! 
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